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Preface 


This  short  paper  seeks  to  extend  findings  from  the  previously  published  RAND  monograph 
Victory  Has  a  Thousand  Fathers:  Sources  of  Success  in  Counterinsurgency  to  ongoing  operations 
in  Afghanistan.  The  core  of  the  research  involved  designing  and  conducting  an  expert  elicita¬ 
tion  exercise  (using  classic  Delphi  methods)  to  complete  the  counterinsurgency  (COIN)  score- 
card  developed  as  part  of  the  earlier  study,  extending  its  application  to  ongoing  operations  in 
Afghanistan.  The  expert  elicitation  was  completed  in  January  and  February  2011. 

This  work  will  be  of  interest  to  defense  analysts  and  military  planners  who  are  responsible 
for  evaluating  current  U.S.  operations  and  COIN  approaches,  particularly  in  Afghanistan;  to 
military  and  civilian  decisionmakers  with  responsibility  for  Afghanistan;  to  academics  and 
scholars  who  engage  in  historical  research  of  COIN,  insurgency,  and  irregular  warfare;  and  to 
students  of  contemporary  and  historical  international  conflicts. 

Readers  will  also  find  the  following  RAND  publications  to  be  of  interest: 

•  Victory  Has  a  Thousand  Fathers:  Sources  of  Success  in  Counterinsurgency ,  by  Christopher 

Paul,  Colin  P.  Clarke,  and  Beth  Grill  (MG-964-OSD) 

•  Victory  Has  a  Thousand  Fathers:  Detailed  Counterinsurgency  Case  Studies ,  by  Christopher 

Paul,  Colin  P.  Clarke,  and  Beth  Grill  (MG-964/1-OSD). 

This  research  was  conducted  within  the  International  Security  and  Defense  Policy  Center 
of  the  RAND  National  Defense  Research  Institute,  a  federally  funded  research  and  develop¬ 
ment  center  sponsored  by  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the  Unified 
Combatant  Commands,  the  Navy,  the  Marine  Corps,  the  defense  agencies,  and  the  defense 
Intelligence  Community. 

For  more  information  on  the  International  Security  and  Defense  Policy  Center,  see 
http://www.rand.org/nsrd/ndri/centers/isdp.html  or  contact  the  director  (contact  information 
is  provided  on  the  web  page). 


Abstract 


The  previously  published  RAND  monograph  Victory  Has  a  Thousand  Fathers:  Sources  of  Suc¬ 
cess  in  Counterinsurgency  used  detailed  case  studies  of  the  30  insurgencies  worldwide  begun 
and  completed  between  1978  and  2008  to  analyze  correlates  of  success  in  COIN.  One  of  the 
core  findings  of  that  effort  was  that  a  case’s  score  on  a  scorecard  of  15  good  factors  or  practices 
minus  12  bad  factors  or  practices  perfectly  predicted  case  outcomes  in  the  data.  Cases  in  which 
COIN  forces  were  able  to  maximize  the  presence  of  good  factors  and  minimize  the  presence 
of  bad  factors  resulted  in  COIN  force  success.  Specifically,  cases  with  a  good-minus-bad  score 
of  +5  or  greater  were  always  won  by  the  government,  and  cases  with  a  good-minus-bad  score  of 
zero  or  lower  were  always  won  by  the  insurgents.1 

The  current  research  effort  involved  developing  and  conducting  an  expert  elicitation  exer¬ 
cise  to  complete  the  scorecard  for  operations  in  Afghanistan  in  early  2011.  A  panel  of  11 
experts  on  Afghanistan  were  asked  to  make  “worst-case”  assessments  of  the  scorecard  factors. 
Based  on  consensus  results  for  the  scorecard,  early  2011  Afghanistan  scores  +3.5.  This  score  is 
lower  than  the  lowest-scoring  COIN  win  in  the  past  30  years  (which  was  +5),  but  it  is  higher 
than  the  highest-scoring  loss  (which  was  0).  This  highlighted  that  certain  factors  are  absent 
whose  presence  would  likely  increase  the  prospects  for  success. 

Specifically,  certain  government  legitimacy  and  good  governance  factors  were  scored  as 
missing.  Perhaps  of  greatest  concern  are  scores  indicating  the  failure  to  date  to  substantially 
disrupt  the  fulfillment  of  the  insurgents’  tangible  support  needs.  In  the  30  historical  cases,  all 
the  COIN  winners  did  this  while  none  of  the  losers  were  able  to.  This  analysis  recommends 
seeking  further  improvements  to  increase  key  good  factors  and  decrease  bad  factors  in  order  to 
increase  confidence  in  the  prospects  for  and  likelihood  of  victory  in  Afghanistan. 


1  Perfect  prediction  in  the  30  historical  cases  is  by  no  means  a  guarantee  of  perfect  prediction  across  all  possible  cases.  Full 
details  of  the  historical  research  can  be  found  in  Christopher  Paul,  Colin  P.  Clarke,  and  Beth  Grill,  Victory  Has  a  Thoiisand 
Fathers:  Sources  of  Success  in  Counterinsurgency ,  Santa  Monica,  Calif.:  RAND  Corporation,  MG-964-OSD,  2010b.  For  the 
full  case  studies,  see  Christopher  Paul,  Colin  P.  Clarke,  and  Beth  Grill,  Victory  Has  a  Thousand  Fathers:  Detailed  Counter¬ 
insurgency  Case  Studies,  Santa  Monica,  Calif.:  RAND  Corporation,  MG-964/1-OSD,  2010a. 
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Findings  from  Previous  Research  on  30  Years  of  Insurgency1 


The  previously  published  RAND  monograph  Victory  Has  a  Thousand  Fathers:  Sources  of  Suc¬ 
cess  in  Counterinsurgency  used  detailed  case  studies  of  the  30  insurgencies  begun  and  com¬ 
pleted  worldwide  between  1978  and  2008  to  analyze  correlates  of  success  in  counterinsurgency 
(COIN).  That  study  produced  six  key  findings  regarding  success  in  COIN  in  recent  history: 

•  Effective  COIN  practices  tend  to  run  in  packs. 

•  The  balance  of  good  versus  bad  practices  perfectly  predicts  outcomes. 

•  Of  20  COIN  approaches  tested,  13  received  strong  support,  while  three  were  not  sup¬ 
ported  by  the  evidence. 

•  Repression  wins  phases  but  usually  not  cases. 

•  With  regard  to  insurgent  success,  tangible  support  trumps  popular  support. 

•  Poor  beginnings  do  not  necessarily  lead  to  poor  ends. 

A  more  detailed  explanation  of  these  findings  can  be  found  in  the  earlier  monograph.  The 
current  study  focused  on  the  importance  of  maximizing  good  factors  and  minimizing  bad  fac¬ 
tors  in  COIN,  and  it  sought  to  assess  the  balance  of  these  factors  in  Afghanistan  in  early  2011. 


The  Counterinsurgency  Scorecard 

As  mentioned,  one  of  the  core  findings  of  the  previous  effort  was  that  a  case’s  score  on  a  score- 
card  of  15  equally  weighted  good  factors  or  practices  minus  12  equally  weighted  bad  factors  or 
practices  perfectly  predicted  case  outcomes  in  the  data.  Table  1  lists  the  good  and  the  bad  factors. 

Over  the  30  cases,  taking  the  sum  of  the  good  minus  the  bad  reveals  that  cases  with  a 
good-minus-bad  score  of  +5  or  greater  were  always  won  by  the  government,  and  cases  with 
a  good-minus-bad  score  of  zero  or  lower  were  always  won  by  the  insurgents.2  In  other  words, 
scores  on  the  scorecard  perfectly  discriminated  the  historical  cases  into  wins  and  losses.3  The 


1  The  author  would  like  to  thank  John  Gordon  IV  at  RAND  and  Joseph  Collins  at  the  National  Defense  University  for 
their  insightful  comments  on  an  earlier  draft  as  part  of  RAND’s  quality  assurance  process.  Thanks,  too,  are  owed  to  the 
members  of  the  anonymous  expert  panel  for  their  participation  in  the  Delphi  exercise. 

2  Full  details  can  be  found  Christopher  Paul,  Colin  P.  Clarke,  and  Beth  Grill,  Victory  Has  a  Thousand  Fathers:  Sources  of 
Success  in  Counterinsurgency ,  Santa  Monica,  Calif.:  RAND  Corporation,  MG-964-OSD,  2010b. 

3  Note  that  perfect  discrimination  in  these  30  cases  does  not  guarantee  perfect  discrimination  across  all  possible  cases. 
With  an  increased  sample  size  (additional  cases  and  data),  it  is  possible — even  likely — that  the  estimated  model  would 
change  slightly. 
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2  Counterinsurgency  Scorecard:  Afghanistan  in  Early  2011  Relative  to  the  Insurgencies  of  the  Past  30  Years 


Table  1 

"Good"  and  "Bad"  COIN  Factors  and  Practices 


15  Good  COIN  Practices  12  Bad  COIN  Practices 


The  COIN  force  realized  at  least  two  strategic 
communication  factors. 

The  COIN  force  reduced  at  least  three  tangible  support 
factors. 

The  government  realized  at  least  two  government 
legitimacy  factors. 

The  government  realized  at  least  one  democracy 
factor. 

The  COIN  force  realized  at  least  one  intelligence  factor. 

The  COIN  force  was  of  sufficient  strength  to  force  the 
insurgents  to  fight  as  guerrillas. 

The  government/state  was  competent. 

The  COIN  force  avoided  excessive  collateral  damage, 
disproportionate  use  of  force,  or  other  illegitimate 
applications  of  force. 

The  COIN  force  sought  to  engage  and  establish  positive 
relations  with  the  population  in  the  area  of  conflict. 

Short-term  investments,  improvements  in  infrastructure 
or  development,  or  property  reform  occurred  in  the  area 
of  conflict  controlled  or  claimed  by  the  COIN  force. 

The  majority  of  the  population  in  the  area  of  conflict 
supported  or  favored  the  COIN  force. 

The  COIN  force  established  and  then  expanded  secure 
areas. 

The  COIN  force  had  and  used  uncontested  air 
dominance. 

The  COIN  force  provided  or  ensured  the  provision  of 
basic  services  in  areas  that  it  controlled  or  claimed  to 
control. 

The  perception  of  security  was  created  or  maintained 
among  the  population  in  areas  that  the  COIN  force 
claimed  to  control. 


The  COIN  force  used  both  collective  punishment  and 
escalating  repression. 

The  primary  COIN  force  was  perceived  to  be  an 
external  occupier. 

COIN  force  or  government  actions  contributed  to 
substantial  new  grievances  claimed  by  the  insurgents. 

Militias  worked  at  cross-purposes  with  the  COIN  force 
or  government. 

The  COIN  force  resettled  or  removed  civilian 
populations  for  population  control. 

COIN  force  collateral  damage  was  perceived  by  the 
population  in  the  area  of  conflict  as  worse  than  the 
insurgents'. 

In  the  area  of  conflict,  the  COIN  force  was  perceived  as 
worse  than  the  insurgents. 

The  COIN  force  failed  to  adapt  to  changes  in  adversary 
strategy,  operations,  or  tactics. 

The  COIN  force  engaged  in  more  coercion  or 
intimidation  than  the  insurgents. 

The  insurgent  force  was  individually  superior  to  the 
COIN  force  by  being  either  more  professional  or  better 
motivated. 

The  COIN  force  or  its  allies  relied  on  looting  for 
sustainment. 

The  COIN  force  and  government  had  different  goals  or 
level  of  commitment. 


lesson  for  current  or  future  counterinsurgents  is  obvious:  Maximizing  the  presence  of  good 
factors  and  minimizing  the  presence  of  bad  factors  will  lead  to  the  best  chances  for  success. 

Table  2  lists  the  30  insurgencies  that  began  and  concluded  between  1978  and  2008. 4 
Table  2  also  lists  for  each  case  the  sum  of  the  good  factors  (of  a  possible  15,  listed  in  Table  1), 
the  sum  of  the  bad  factors  (of  a  possible  12,  listed  in  Table  1),  the  net  of  good-minus-bad, 
and  the  outcome  of  the  case  (either  a  government  loss  or  a  government  win).5 


4  For  full  details  on  case  selection  and  the  process  of  data  collection  and  factor  scoring  for  each  case,  see  Paul,  Clarke,  and 
Grill,  2010b. 

5  Note  that  “loss”  also  includes  outcomes  assessed  as  “mixed,  favoring  insurgents,”  and  “win”  includes  outcomes  assessed 
as  “mixed,  favoring  the  COIN  force.” 
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Table  2 

Cases  and  Scorecard  Scores 


Case 

Years 

Good  Factors 
(of  15) 

Bad  Factors 
(of  12) 

Good  -  Bad 
Factors 

Outcome 

Afghanistan  (post-Soviet) 

1992-1996 

0 

10 

-10 

Loss 

Somalia 

1980-1991 

1 

10 

-9 

Loss 

Chechnya  1 

1994-1996 

2 

10 

-8 

Loss 

Rwanda 

1990-1994 

2 

10 

-8 

Loss 

Zaire  (anti-Mobutu) 

1996-1997 

0 

8 

-8 

Loss 

Nicaragua  (Somoza) 

1978-1979 

0 

8 

-8 

Loss 

Sudan  (SPLA) 

1984-2004 

2 

9 

-7 

Loss 

Kosovo 

1996-1999 

1 

8 

-7 

Loss 

Afghanistan  (anti-Soviet) 

1978-1992 

1 

7 

-6 

Loss 

Papua  New  Guinea 

1988-1998 

3 

9 

-6 

Loss 

Burundi 

1993-2003 

2 

8 

-6 

Loss 

Bosnia 

1992-1995 

1 

6 

-5 

Loss 

Moldova 

1990-1992 

2 

6 

-4 

Loss 

Georgia/Abkhazia 

1992-1994 

1 

5 

-4 

Loss 

Liberia 

1989-1997 

3 

7 

-4 

Loss 

Afghanistan  (Taliban) 

1996-2001 

2 

6 

-4 

Loss 

Nagorno-Karabakh 

1992-1994 

1 

4 

-3 

Loss 

DR  Congo  (anti-Kabila) 

1998-2003 

1 

4 

-3 

Loss 

Tajikistan 

1992-1997 

2 

5 

-3 

Loss 

Kampuchea 

1978-1992 

1 

3 

-2 

Loss 

Nepal 

1997-2006 

3 

5 

-2 

Loss 

Nicaragua  (Contras) 

1981-1990 

4 

4 

0 

Loss 

Croatia 

1992-1995 

8 

3 

+5 

Win 

Turkey  (PKK) 

1984-1999 

11 

5 

+6 

Win 

Uganda  (ADF) 

1986-2000 

8 

0 

+8 

Win 

Algeria  (GIA) 

1992-2004 

9 

1 

+8 

Win 

El  Salvador 

1979-1992 

12 

2 

+10 

Win 

Peru 

1980-1992 

13 

2 

+11 

Win 

Senegal 

1982-2002 

13 

0 

+13 

Win 

Sierra  Leone 

1991-2002 

14 

1 

+13 

Win 

Scoring  Afghanistan  in  2011 

In  addition  to  providing  an  interesting  summary  of  historical  results,  the  scorecard  is  meant  to 
be  a  useful  prognostication  tool  for  current  and  future  COIN  efforts.  Without  asserting  that 
the  scorecard’s  ability  to  discriminate  wins  from  losses  perfectly  over  the  period  1978-2008 
guarantees  its  ability  to  predict  outcomes  in  the  future,  scores  for  a  current  or  future  COIN 
effort  should  be  indicative  of  whether  that  effort  is  on  a  path  likely  to  lead  to  success  or  on  a 
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path  less  likely  to  do  so.  Specific  factors  scored  as  present  or  absent  should  also  provide  guid¬ 
ance  to  decisionmakers,  strategists,  and  planners  about  areas  ripe  for  renewed  emphasis. 

With  that  in  mind,  the  current  research  effort  sought  to  complete  the  scorecard  for  ongo¬ 
ing  operations  in  Afghanistan  as  of  early  2011.  Because  the  assignment  of  specific  scorecard 
factors  as  present  or  absent  is  potentially  both  contentious  and  sensitive,  an  expert  elicitation 
exercise  was  conducted  using  an  anonymous  panel  of  persons  who  are  knowledgeable  about 
ongoing  operations  in  Afghanistan.  Panelists  included  RAND  staff  with  expertise  on  Afghani¬ 
stan  or  who  had  research  experience  with  or  had  deployed  to  Afghanistan,  serving  field-grade 
U.S.  military  officers  with  multiple  (and  recent)  deployments  to  Afghanistan,  military  veter¬ 
ans  who  were  current  on  COIN  research,  university  faculty  members,  journalists,  and  experts 
from  other  prominent  think  tanks.  Full  details  of  the  expert  elicitation  process  are  provided  in 
the  appendix. 

In  keeping  with  the  approach  used  in  the  previous  study  to  quantify  the  30  cases  and 
create  the  scorecard,  participants  were  asked  to  make  worst-case  assessments.  To  be  considered 
present,  positive  factors  were  required  to  be  present  over  the  preponderance  of  the  area  of  con¬ 
flict,  not  just  in  isolated  cases.  Similarly,  bad  factors  were  counted  as  present  if  they  occurred 
with  any  frequency  greater  than  isolated  incidents.  So,  good  factors  needed  to  be  preponder¬ 
antly  present,  but  bad  factors  needed  only  to  be  routinely  present  in  certain  areas  or  among 
certain  segments  of  the  force.  (Further  details  about  the  scoring  instructions  can  be  found  in 
the  appendix.) 

The  expert  elicitation  provided  scores  for  the  51  specific  factors  in  the  scorecard  necessary 
to  calculate  the  15  good  and  12  bad  factors  and  practices  listed  in  Table  l.6  All  51  subfactors 
and  how  they  roll  up  into  the  15  good  and  12  bad  factors  can  been  seen  in  Table  3.  Where 
there  was  consensus  on  the  presence  or  absence  of  a  factor,  it  was  scored  “1”  if  present  or  “0”  if 
absent;  where  there  was  not  consensus  among  the  panelists,  factors  were  scored  “0.5”  for  nei¬ 
ther  present  nor  absent.7  The  detailed  scorecard  and  the  consensus  assessment  of  each  factor 
and  subfactor  are  presented  in  Tables  4  and  5. 

The  scorecard  as  completed  through  expert  elicitation  indicates  that  eight  good  factors 
are  currently  present  in  Afghanistan  and  4.5  bad  scorecard  factors  are  present.  Flalf-points 
result  from  the  panel’s  lack  of  consensus  on  a  factor,  resulting  in  it  being  scored  0.5 — neither 
present  nor  absent.  The  eight  good  factors  include  two  half-points  and  thus  represent  a  range 
from  7  to  9.  The  4.5  bad  factors  include  a  single  half-point,  thus  representing  a  range  between 
4  and  5.  Subtracting  4.5  from  eight  provides  a  scorecard  net  result  of  +3.5.  (With  all  the  half¬ 
points  maximized  in  one  direction  or  the  other,  this  represents  a  range  from  +2  to  +5.) 

Table  3  shows  where  these  scores  (8  good,  4.5  bad,  +3.5  net)  would  fit  in  among  the  30 
insurgencies  that  took  place  between  1978  and  2008.  Having  eight  good  factors  present  is  tied 
for  the  lowest  number  of  good  factors  present  in  a  historical  government  win  but  is  noticeably 
better  than  the  best  score  achieved  in  a  loss  (4).  Having  4.5  bad  factors  is  close  to  the  outcome 
of  the  worst-scoring  government  winner  and  is  more  typical  of  scores  for  cases  in  which  gov¬ 
ernments  lost.  The  net  score,  +3.5,  falls  squarely  within  the  empirical  gap  between  historical 


6  Note  that  it  is  51  factors  rather  than  15  +  12  =  27  factors  because  some  are  summary  factors  (such  as  “COIN  force  real¬ 
izes  at  least  two  strategic  communication  factors”)  that  rely  on  multiple  subordinate  factors  for  calculation. 

7  Consensus,  for  this  exercise,  was  considered  to  be  agreement  among  at  least  70  percent  of  panelists.  In  practice,  for  most 
consensus  factors,  the  degree  of  accord  was  considerably  higher.  The  raw  averages  (equivalent  to  percent  agreement)  are 
listed  in  Table  A.l  in  the  appendix. 
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Table  3 

Cases  and  Scores  from  the  Study  of  30  Historical  Insurgencies,  with  Early  2011  Afghanistan  Scores 


Case 

Years 

Good  Factors 
(of  15) 

Bad  Factors 
(of  12) 

Good  -  Bad 
Factors 

Outcome 

Afghanistan  (post-Soviet) 

1992-1996 

0 

10 

-10 

Loss 

Somalia 

1980-1991 

1 

10 

-9 

Loss 

Chechnya  1 

1994-1996 

2 

10 

-8 

Loss 

Rwanda 

1990-1994 

2 

10 

-8 

Loss 

Zaire  (anti-Mobutu) 

1996-1997 

0 

8 

-8 

Loss 

Nicaragua  (Somoza) 

1978-1979 

0 

8 

-8 

Loss 

Sudan  (SPLA) 

1984-2004 

2 

9 

-7 

Loss 

Kosovo 

1996-1999 

1 

8 

-7 

Loss 

Afghanistan  (anti-Soviet) 

1978-1992 

1 

7 

-6 

Loss 

Papua  New  Guinea 

1988-1998 

3 

9 

-6 

Loss 

Burundi 

1993-2003 

2 

8 

-6 

Loss 

Bosnia 

1992-1995 

1 

6 

-5 

Loss 

Moldova 

1990-1992 

2 

6 

-4 

Loss 

Georgia/Abkhazia 

1992-1994 

1 

5 

-4 

Loss 

Liberia 

1989-1997 

3 

7 

-4 

Loss 

Afghanistan  (Taliban) 

1996-2001 

2 

6 

-4 

Loss 

Nagorno-Karabakh 

1992-1? 

At  4.5,  Afghanistan 
(early  2011)  is  close 

4 

-3 

Loss 

DR  Congo  (anti-Kabila) 

1998-21 

to  the  worst-scoring 
winner  but  also  close 

4 

-3 

Loss 

Tajikistan 

1992-1? 

to  many  losers. 

5 

-3 

Loss 

Kampuchea 

1978-1992 

1 

\  3 

-2 

Loss 

Nepal 

1997-2006 

3 

\  5 

-2 

Loss 

Nicaragua  (Contras) 
Afghanistan  (early  2011) 


1981-1990 


0 

+3.5 


At  +3.5, 
Afghanistan 
(early  2011)  is  in 


Croatia 

1992-1995 

8 

\  3 

+5 

between  winners 

Turkey  (PKK) 

1984-1999 

11 

^5 

+6 

and  losers  over  the 
past  30  years. 

Uganda  (ADF) 

At  8,  Afghanistan 

8 

0 

+8 

Win 

Algeria  (GIA) 

(early  2011)  is  tied 

with  the  lowest- 

9 

1 

+8 

Win 

El  Salvador 

scoring  winner. 

12 

2 

+10 

Win 

Peru 

1980-1992 

13 

2 

+11 

Win 

Senegal 

1982-2002 

13 

0 

+13 

Win 

Sierra  Leone 

1991-2002 

14 

1 

+13 

Win 

winners  and  losers  (between  0  and  +5),  making  it  unclear  which  of  those  two  outcomes  is  most 
likely. 

The  results  from  the  previous  study  suggest  that  8  good  factors  is  good  news,  4.5  bad 
factors  is  marginal,  and  +3.5  net  is  indeterminate.  From  a  policy  perspective,  this  recommends 
seeking  further  improvements  to  increase  good  factors  and  decrease  bad  factors  in  order  to  increase 
confidence  and  the  likelihood  of  victory  in  Afghanistan. 
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Of  course,  the  general  findings  of  the  previous  RAND  study  make  this  a  general  recom¬ 
mendation;  COIN  forces  should  always  seek  to  maximize  good  factors  and  minimize  bad  fac¬ 
tors.  Further  value  of  this  scorecard  and  elicitation  comes  from  the  specific  factors  deemed  pres¬ 
ent  or  absent.  Specific  factors  and  related  discussion  (along  with  suggestions  regarding  which 
factors  might  be  improved  upon)  are  presented  in  subsequent  sections.  Byway  of  preview,  criti¬ 
cal  factors  that  are  absent  or  only  marginally  present  but  ripe  for  improvement  include  govern¬ 
ment  reform/improvement  (in  terms  of  legitimacy,  state  competence,  governance,  and  services 
provided),  security  (success  and  consistency  in  establishing  and  expanding  secure  areas,  cre¬ 
ation  and  maintenance  of  a  perception  of  security  throughout  the  area  of  conflict),  and  inter¬ 
diction  of  tangible  support  to  the  insurgents  (interrupting  flows  of  materiel  and  other  forms  of 
support  both  from  within  Afghanistan  and  across  borders). 


Detailed  Factors  in  the  Current  Case 

Tables  4  and  5  present  all  factors  and  subfactors  in  the  scorecard  and  the  factors  scored  as  pres¬ 
ent  or  absent  in  the  RAND  Afghanistan  Delphi  exercise.  In  each  row,  a  “1”  indicates  that  the 
consensus  view  was  that  the  factor  or  practice  was  deemed  to  be  present  in  the  current  con¬ 
flict;  a  “0”  indicates  that  the  consensus  view  was  that  it  was  absent.  When  viewing  the  scores, 
remember  that  participants  were  asked  to  make  worst-case  assessments;  isolated  anecdotes  or 
limited  accounts  of  success  in  these  areas  were  not  considered  sufficient  grounds  to  score  a 
factor  (be  it  a  good  factor  or  a  bad  factor)  as  present.  Where  the  score  is  “0.5,”  there  was  no  clear 
consensus  among  participants.  These  factors  are  discussed  in  greater  detail  later.  Rows  begin¬ 
ning  with  numbers  are  primary  factors  (i.e.,  the  top-level  factors  listed  Table  1);  rows  beginning 
with  lowercase  letters  are  the  subfactors  that  constitute  the  primary  factors. 

Critical  Factors 

The  discussion  and  findings  from  the  previous  research  suggest  that  some  of  the  good  factors 
that  the  participants  identified  as  absent  are  particularly  concerning. 

Early  2011  Afghanistan  was  scored  as  having  eight  of  a  possible  15  good  factors  present 
and  thus  seven  (actually,  six  and  two  half-points)  absent.  The  absent  factors  are  as  follows: 

•  COIN  force  realizes  at  least  two  strategic  communication  factors 

•  COIN  force  reduces  at  least  three  tangible  support  factors 

•  Government  realizes  at  least  two  government  legitimacy  factors 

•  Government/state  is  competent 

•  Majority  of  population  in  area  of  conflict  supports/favors  COIN  forces  (0.5) 

•  COIN  force  establishes  and  then  expands  secure  areas  (0.5) 

•  COIN  force  provides  or  ensures  provision  of  basic  services  in  areas  it  controls  or  claims 
to  control 

•  Perception  of  security  created  or  maintained  among  populations  in  areas  COIN  force 
claims  to  control. 

These  absences  suggest  three  areas  for  improvement.  First,  in  terms  of  government,  legiti¬ 
macy,  state  competence,  governance/services,  and  the  support  of  the  population  were  all  scored 
as  lacking.  Improving  the  legitimacy  of  the  Afghan  government  in  the  eyes  of  its  people  and 
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Table  4 

Good  Factors  Present  in  Early  2011  Afghanistan  (Total:  8  of  15) 


Good  Factors 

1.  COIN  force  realizes  at  least  two  strategic  communication  factors  (Score  1  if  sum  of  a  through  g 
is  at  least  2) 

a.  COIN  force  and  government  actions  consistent  with  messages  (delivering  on  promises) 

(Score  1  if  YES) 

b.  COIN  force  maintains  credibility  with  population  in  the  area  of  conflict  (includes  expectation 
management)  (Score  1  if  YES) 

c.  Messages/themes  coherent  with  overall  COIN  approach  (Score  1  if  YES) 

d.  COIN  force  avoids  creating  unattainable  expectations  (Score  1  if  YES) 

e.  Themes  and  messages  coordinated  for  all  involved  government  agencies  (Score  1  if  YES) 

f.  Earnest  IO/PSYOP/strategic  communication/messaging  effort  (Score  1  if  YES) 

g.  Unity  of  effort/unity  of  command  maintained  (Score  1  if  YES) 

2.  COIN  force  reduces  at  least  three  tangible  support  factors  (Score  1  if  sum  of  a  through  j 
is  at  least  3) 

a.  Flow  of  cross-border  insurgent  support  significantly  decreased,  remains  dramatically 
reduced,  or  largely  absent  (Score  1  if  YES) 

b.  Important  external  support  to  insurgents  significantly  reduced  (Score  1  if  YES) 

c.  Important  internal  support  to  insurgents  significantly  reduced  (Score  1  if  YES) 

d.  Insurgents'  ability  to  replenish  resources  significantly  diminished  (Score  1  if  YES) 

e.  Insurgents  unable  to  maintain  or  grow  force  size  (Score  1  if  YES) 

f.  COIN  force  efforts  resulting  in  increased  costs  for  insurgent  processes  (Score  1  if  YES) 

g.  COIN  forces  effectively  disrupt  insurgent  recruiting  (Score  1  if  YES) 

h.  COIN  forces  effectively  disrupt  insurgent  materiel  acquisition  (Score  1  if  YES) 

i.  COIN  forces  effectively  disrupt  insurgent  intelligence  (Score  1  if  YES) 

j.  COIN  forces  effectively  disrupt  insurgent  financing  (Score  1  if  YES) 

3.  Government  realizes  at  least  two  government  legitimacy  factors  (Score  1  if  sum  of  a  through  e 
is  at  least  2) 

a.  Government  corruption  reduced/good  governance  increased  since  onset  of  conflict 
(Score  1  if  YES) 

b.  Government  leaders  selected  in  a  manner  considered  just  and  fair  by  majority  of  population 
in  area  of  conflict  (Score  1  if  YES) 

c.  Majority  of  citizens  in  the  area  of  conflict  view  government  as  legitimate  (Score  1  if  YES) 
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Table  4 — Continued 


Good  Factors 


d.  Government  provides  better  governance  than  insurgents  in  area  of  conflict  (Score  1  if  YES) 


e.  COIN  force  provides  or  ensures  provision  of  basic  services  in  areas  it  controls  or  claims  to 
control  (Score  1  if  YES) 

4.  Government  realizes  at  least  one  democracy  factor  (Score  1  if  sum  of  a  through  d  is  at  least  1) 

a.  Government  a  functional  democracy  (Score  1  if  YES) 

b.  Government  a  partial  or  transitional  democracy  (Score  1  if  YES) 

c.  Free  and  fair  elections  held  (Score  1  if  YES) 


d.  Government  respects  human  rights  and  allows  free  press  (Score  1  if  YES) 

5.  COIN  force  realizes  at  least  one  intelligence  factor  (Score  1  if  sum  of  a  and  b  is  at  least  1) 


a.  Intelligence  adequate  to  support  kill/capture  or  engagements  on  COIN  force's  terms 
(Score  1  if  YES) 

b.  Intelligence  adequate  to  allow  COIN  force  to  disrupt  insurgent  processes  or  operations 
(Score  1  if  YES) 

6.  COIN  force  of  sufficient  strength  to  force  insurgents  to  fight  as  guerrillas  (Score  1  if  YES) 

7.  Government/state  is  competent  (Score  1  if  YES) 

8.  COIN  force  avoids  excessive  collateral  damage,  disproportionate  use  of  force,  or  other 
illegitimate  applications  of  force  (Score  1  if  YES) 

9.  COIN  force  seeks  to  engage  and  establish  positive  relations  with  population  in  area  of  conflict 
(Score  1  if  YES) 

10.  Short-term  investments,  improvements  in  infrastructure/development,  or  property  reform 
in  area  of  conflict  controlled  or  claimed  by  COIN  force  (Score  1  if  YES) 

11.  Majority  of  population  in  area  of  conflict  supports/favors  COIN  forces  (Score  1  if  YES) 

12.  COIN  force  establishes  and  then  expands  secure  areas  (Score  1  if  YES) 

13.  COIN  force  has  and  uses  uncontested  air  dominance  (Score  1  if  YES) 


14.  COIN  force  provides  or  ensures  provision  of  basic  services  in  areas  it  controls  or  claims  to 
control  (Score  1  if  YES) 

15.  Perception  of  security  being  created  or  maintained  among  populations  in  areas  COIN  force 
claims  to  control  (Score  1  if  YES) 


NOTE:  IO  =  information  operations.  PSYOP  =  psychological  operations. 
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Table  5 

Bad  Factors  Present  in  Early  2011  Afghanistan  (Total:  4.5  of  12) 


Bad  Factors 


1.  COIN  force  uses  both  collective  punishment  and  escalating  repression  (Score  1  if  sum  of  a  and  b 
is  at  least  1) 

a.  COIN  force  employs  escalating  repression  (Score  1  if  YES) 

b.  COIN  force  employs  collective  punishment  (Score  1  if  YES) 

2.  Primary  COIN  force  perceived  to  be  an  external  occupier  (Score  1  if  YES) 

3.  COIN  force  or  government  actions  contribute  to  substantial  new  grievances  claimed  by  the 
insurgents  (Score  1  if  YES) 

4.  Militias  work  at  cross-purposes  with  COIN  force/government  (Score  1  if  YES) 

5.  COIN  force  resettles/removes  civilian  populations  for  population  control  (Score  1  if  YES) 

6.  COIN  force  collateral  damage  perceived  by  population  in  area  of  conflict  as  worse  than 
insurgents'  (Score  1  if  YES) 

7.  In  area  of  conflict,  COIN  force  perceived  as  worse  than  insurgents  (Score  1  if  YES) 

8.  COIN  force  fails  to  adapt  to  changes  in  adversary  strategy,  operations,  or  tactics  (Score  1  if  YES) 

9.  COIN  force  engages  in  more  coercion/intimidation  than  insurgents  (Score  1  if  YES) 

10.  Insurgent  force  individually  superior  to  COIN  force  by  being  either  more  professional  or  better 
motivated  (Score  1  if  YES) 

11.  COIN  force  or  allies  rely  on  looting  for  sustainment  (Score  1  if  YES) 

12.  COIN  force  and  government  have  different  goals/level  of  commitment  (Score  1  if  YES) 


0 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0.5 

1 

improving  the  services  and  governance  provided  could  dramatically  improve  the  situation 
there. 

Second,  in  terms  of  security,  the  forces  of  order  are  already  attempting  to  establish  and 
then  expand  secure  areas,  though  the  expert  panel  was  mixed  regarding  the  success  of  those 
efforts  (see  the  discussion  in  the  next  section).  If  that  “0.5”  could  be  converted  to  “1,”  then  the 
“perception  of  security”  factor  might  also  become  “1,”  again  improving  the  situation  (and 
the  scorecard). 

The  third  area  for  improvement  is  the  interdiction  of  tangible  support.  This  concerns  the 
extent  to  which  COIN  forces  have  cut  flows  of  support  (e.g.,  materiel,  personnel,  financial, 
intelligence)  to  the  insurgents.  The  expert  panel  scored  only  one  of  the  ten  tangible  support 
reduction  factors  as  present;  to  be  scored  “1”  on  the  scorecard,  at  least  three  must  be  present 
(and  obviously  even  more  would  be  better).  This  is  particularly  concerning,  because  in  the 
30  historical  cases,  all  of  the  COIN  winners  disrupted  at  least  three  tangible  support  factors 
and  none  of  the  losers  did.  The  discussion  revealed  that  the  insurgents  meet  their  tangible  sup¬ 
port  needs  using  many  sources,  none  of  which  has  been  significantly  disrupted:  supporters 
across  the  border  in  Pakistan  (and  in  other  countries),  profits  from  drug  trafficking  and  other 
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criminal  activities,  and  supportive  populations  inside  Afghanistan.  Disrupting  tangible  sup¬ 
port  would  require  significant  efforts  to  interrupt  support  flows  from  all  of  these  sources,  but 
it  may  well  be  necessary  for  COIN  forces  to  prevail. 

Four  (and  one  half-point)  of  the  12  bad  factors  were  scored  as  present  by  the  expert  panel: 

•  Primary  COIN  force  perceived  to  be  an  external  occupier 

•  COIN  force  or  government  actions  contribute  to  substantial  new  grievances  claimed  by 

the  insurgents 

•  Militias  work  at  cross-purposes  with  COIN  force/government 

•  COIN  force  or  allies  rely  on  looting  for  sustainment  (0.5) 

•  COIN  force  and  government  have  different  goals/level  of  commitment. 

Of  these  five  factors,  most  are  things  that  the  International  Security  Assistance  Force 
(ISAF)  can  do  little  about,  directly.  “Primary  COIN  force  perceived  to  be  an  external  occu¬ 
pier”  will  not  go  away  until  ISAF  draws  down  considerably  and  the  Afghan  National  Secu¬ 
rity  Forces  substantially  increase  in  size  and  capability.8  “Militias  work  at  cross-purposes  with 
COIN  force/government,”  “COIN  force  or  allies  rely  on  looting  for  sustainment,”  and  “COIN 
force  and  government  have  different  goals/level  of  commitment”  are  all  squarely  on  the  Afghan 
government  or  the  Afghan  forces.  “COIN  force  or  government  actions  contribute  to  substan¬ 
tial  new  grievances”  might  be  more  tractable,  but  that  depends  in  part  on  Afghan  government 
decisions  and  in  part  on  the  capabilities  available  (air  strikes  being  a  core  source  of  complaint). 

This  suggests  that  ISAF  would  do  well  to  focus  on  increasing  the  good  factors  that  are 
present  (as  discussed  earlier),  ensuring  that  the  bad  factors  that  are  absent  stay  that  way,  and 
encouraging  the  Afghan  government  to  increase  the  good  factors  under  its  control  and  decrease 
the  bad  factors  under  its  control. 

Notes  on  Factors  for  Which  Consensus  Was  Lacking 

Although  the  participants  in  the  expert  elicitation  had  or  achieved  consensus  on  most  factors, 
some  factors  remained  sources  of  disagreement  throughout  the  exercise.  Several  of  them  merit 
specific  mention. 

The  first  such  factor  was  “Messages/themes  coherent  with  overall  COIN  approach.”  This 
factor  is  notable  not  so  much  for  the  discussion  (during  which  there  was  genuine  disagreement) 
but  for  its  impact  on  the  scorecard.  Had  this  factor  been  consensually  accepted  as  present 
(scored  “1”),  then  the  superordinate  factor,  “COIN  force  realizes  at  least  two  strategic  com¬ 
munication  factors,”  would  have  been  minimally  satisfied  and  also  would  have  been  scored 
“1.”  Remember,  however,  that  the  goal  of  a  COIN  strategy  should  not  be  to  cross  minimum 
thresholds  on  a  scorecard  but  to  synchronize  a  coordinated  campaign  that  maximizes  good 
factors  and  minimizes  bad  factors.  To  maximize  effectiveness  in  the  area  of  strategic  communi¬ 
cation,  COIN  forces  will  not  only  need  to  more  firmly  establish  the  presence  of  more  strategic 
communication-related  factors  but,  as  noted  earlier,  they  would  also  benefit  from  improving 
the  underlying  conditions  that  inform  the  themes  and  messages  communicated — namely,  gov¬ 
ernment  legitimacy  and  security. 


8  Note  that  the  Afghan  National  Security  Forces  have  increased  substantially  in  size  in  the  past  year.  However,  as  long  as 
international  forces  are  still  in  country  in  substantial  numbers,  the  perception  that  they  constitute  an  occupying  force  will 
likely  remain. 
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More  contentious  than  the  lack  of  consensus  over  the  connection  between  themes/ 
messages  and  the  overall  COIN  approach  in  Afghanistan  was  disagreement  over  “Government 
respects  human  rights  and  allows  free  press.”  Disagreement  here  hinged  primarily  on  issues  of 
women’s  rights  and  religious  freedom.  Participants  broadly  agreed  that  women’s  rights  have 
improved,  as  have  press  freedoms  and  freedom  from  extrajuridical  detentions  and  killings. 
Those  who  scored  this  factor  as  present  emphasized  improvement,  expanded  press  freedom, 
and  limited  extrajudicial  killings  and  detentions  on  the  part  of  government  or  coalition  forces. 
Those  who  indicated  that  this  factor  was  absent  emphasized  the  insufficiency  of  improvement 
and  the  failure  to  meet  global  standards  regarding  women’s  rights  and  religious  freedom.  From 
a  scorecard  perspective,  consensus  on  whether  this  factor  is  present  or  absent  does  not  change 
the  net  score,  as  the  superordinate  factor,  “Government  realizes  at  least  one  democracy  factor,” 
is  satisfied  by  Afghanistan’s  partial  democracy,  independent  of  this  factor.  Again,  note  that  the 
objective  should  be  to  maximize  good  factors  and  minimize  bad  ones,  not  achieve  (or  accept) 
minimal  threshold  levels  in  pursuit  of  a  higher  score. 

Another  hotly  contested  factor  was  “Majority  of  population  in  area  of  conflict  supports/ 
favors  COIN  forces.”  Discussion  surrounding  this  topic  was  particularly  intense,  with  several 
participants  citing  different  public  opinion  polls,  other  participants  disparaging  poll  results  in 
general,  and  significant  mention  of  regional  variation  in  perceptions.  One  of  the  most  strik¬ 
ing  observations  (and  one  about  which  there  appeared  to  be  consensus)  was  that  “the  Afghan 
people  hate  the  Taliban  but  they  equally  despise  the  government.”  With  significant  agreement 
on  that  part  of  the  issue,  the  discussion  then  focused  on  arguments  in  favor  of  the  lesser  of  two 
evils,  and  consensus  was  not  reached. 

Even  if  the  government  and  the  forces  of  order  are  viewed  as  the  lesser  of  two  evils,  being 
minimally  favored  does  not  imply  the  kind  of  popular  engagement  and  popular  support  that 
is  viewed  as  essential  by  advocates  of  population-centric  COIN  (and  as  validated  in  the  earlier 
RAND  study).  This  is  an  area  that  is  ripe  for  improvement  going  forward. 

Another  disputed  factor  was  “COIN  force  establishes  and  then  expands  secure  areas.” 
Panelists  agreed  that  COIN  forces  are  trying  to  do  this,  but  they  were  split  on  their  views  of  the 
success  of  those  efforts.  One  participant  noted,  “They  are  doing  this  in  some  areas,  but  secure 
areas  are  contracting  in  other  areas.”  According  to  another,  “Forces  of  order  establish  a  secure 
area  and,  when  they  depart  or  transition  the  area  to  government  forces,  lose  the  area  to  insur¬ 
gents.”  The  missing  piece,  the  critics  on  the  panel  asserted,  is  the  ability  to  transition  cleared 
areas  to  the  authority  and  supervision  of  the  indigenous  civilian  government  and  police — a 
step,  they  emphasized,  that  is  not  being  taken  and  is  not  likely  to  be  taken  in  the  near  future. 

The  final  contentious  factor  was  from  among  the  12  bad  factors,  “COIN  force  or  allies 
rely  on  looting  for  sustainment.”  None  of  the  participants  asserted  that  members  of  ISAF  were 
engaged  in  looting,  but  several  stated  that  some  “taxation”  occurs  as  part  of  the  corruption  of 
the  Afghan  National  Security  Forces,  including  the  army  and  the  police.  The  dispute,  then, 
involved  three  issues:  first,  whether  this  behavior  was  sufficiently  widespread  to  count  as  pres¬ 
ent;  second,  whether  such  “taxation”  or  required  bribes  were  equivalent  to  looting;  and  third, 
whether  such  taxation  was  essential  for  these  forces’  sustainment  or  just  supplemental  to  their 
incomes.  Consensus  was  not  reached. 
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Afghanistan  in  Comparison  to  Specific  Historical  Cases 

The  analysis  thus  far  has  compared  Afghanistan  in  early  2011  with  patterns  of  data  observed 
in  the  30  historical  cases  or  with  those  cases  in  the  aggregate.  Is  Afghanistan  particularly  like 
any  specific  case?  Short  answer:  Not  really. 

Afghanistan  in  early  2011  is  not  particularly  similar  in  its  specific  details  to  any  of  the 
30  historical  cases.  In  terms  of  its  good-and-bad-factor  summary,  Afghanistan  (8/4.5)  appears 
closest  to  the  insurgency  in  Croatia  (8/3,  and  a  COIN  win).  Though  similar  in  scorecard  sum¬ 
mary  score,  Croatia  is  not  at  all  similar  in  the  specific  pattern  of  detailed  scorecard  factors  pres¬ 
ent  or  absent,  or  in  the  narrative  of  the  case. 

Considering  all  phases  of  all  cases,9  and  examining  the  extent  to  which  the  51  constituent 
factors  and  subfactors  in  the  scorecard  match  between  cases,  is  a  better  way  to  assess  the  simi¬ 
larity  between  the  historical  cases  and  the  current  Afghanistan  case.  Following  this  procedure 
reveals  again  that  none  of  the  phases  of  historical  cases  is  particularly  similar  to  Afghanistan 
(not  even  the  previous  Afghanistan  cases),  and  no  phase  of  any  case  shares  more  than  39  of  the 
51  detailed  scorecard  subfactors  with  early  2011  Afghanistan. 


Conclusions  and  Recommendations 

Comparing  Afghanistan  in  early  2011  with  the  30  insurgencies  conducted  and  concluded 
between  1978  and  2008  offers  some  support  for  optimism  but  not  unreserved  optimism.  Spe¬ 
cifically,  when  compared  using  a  scorecard  of  15  good  and  12  bad  factors  based  on  previ¬ 
ous  historical  research,  Afghanistan’s  eight  good  factors  present  is  tied  for  the  lowest  number 
of  good  factors  present  in  a  historical  government  win,  but  it  is  noticeably  better  than  the 
best  score  achieved  in  a  loss  (4).  Its  4.5  bad  factors  is  close  to  the  worst-scoring  government 
winner  and  is  more  typical  of  scores  for  governments  that  lost.  Finally,  Afghanistan’s  net  score, 
+3.5,  falls  squarely  within  the  empirical  gap  between  historical  winners  and  losers  (between  0 
and  +5). 

The  scorecard  score  is  an  interim  step  toward  recognizing  the  need  for  further  efforts  and 
identifying  specific  factor  areas  in  which  to  make  enhancements  going  forward.  Considering 
the  specific  good  factors  that  are  absent  and  the  bad  factors  that  are  present  suggests  an  empha¬ 
sis  on  improving  progress  toward  good  factors,  with  three  specific  points  of  focus,  as  discussed 
in  the  section  “Critical  Factors”:  (1)  the  competency,  legitimacy,  and  popular  support  of  the 
Afghan  government;  (2)  security;  and  (3)  disrupting  the  tangible  support  of  the  insurgents. 

Most  of  the  areas  highlighted  for  improvement  are  already  focal  efforts  of  the  coalition 
leadership.  Further,  while  it  is  easy  to  assert  that  these  things  should  be  done,  that  does  not 
make  them  any  easier  to  actually  do;  some  of  these  suggestions  have  been  the  focus  of  ongoing 
efforts  for  some  time  now  and  continue  to  face  tough  challenges.  These  conclusions  provide 
further  support  for  and  endorsement  of  many  command  priorities,  and  the  historical  data  sug¬ 
gest  better  prospects  for  positive  outcomes  if  these  difficult  challenges  can  be  overcome  and 
if  better  legitimacy/governance,  security,  and  disruption  of  tangible  support  can  be  achieved. 


9  In  the  previous  RAND  study,  each  case  was  divided  into  between  two  and  five  phases,  for  a  total  of  86  phases.  In  the 
context  of  this  paper,  data  for  each  case  refer  to  the  decisive  (and  usually  terminal)  phase  of  the  case. 


APPENDIX 


Details  of  the  Expert  Elicitation 


The  Delphi  Method 

The  Delphi  method  was  developed  at  RAND  in  the  1960s.  While  the  technique  has  been 
refined  over  the  years,1  the  fundamental  premise  remains  the  same.  Experts  individually  make 
assessments  or  provide  input  and  then  offer  written  justification  for  those  assessments.  These 
experts  are  then  given  the  opportunity  to  privately  review  the  justifications  offered  by  other 
participants  and  revise  their  assessments  based  on  lines  of  reasoning  that  they  had  failed  to 
include  in  their  own  initial  calculations.  The  result  is  a  consensual  set  of  expert  assessments 
based  on  more  information  than  any  single  expert  initially  considered.  Because  participants 
work  in  private  and  remain  anonymous  to  each  other,  final  evaluations  are  reached  without  any 
of  the  psychological  pitfalls  of  committee  work,  such  as  “specious  persuasion,  the  unwilling¬ 
ness  to  abandon  publicly  expressed  opinions,  and  the  bandwagon  effect  of  majority  opinion.”2 

A  somewhat  pedestrian  example  of  a  Delphi  exercise  demonstrates  this  logic.  Imagine 
that  a  Delphi  exercise  is  convened  to  assure  victory  in  a  carnival  game:  The  investigators  wish 
to  know  how  many  peanuts  are  contained  in  a  large  glass  pig.  A  panel  of  experts  is  assembled, 
including  (among  others)  a  physicist,  a  mathematician,  a  materials  scientist,  a  statistician,  and 
an  expert  in  the  history  of  mountebanks.  Each  performs  his  or  her  calculations  and  generates 
an  estimate  of  the  peanut  content  of  the  pig.  Then,  each  is  asked  to  justify  his  or  her  response, 
explaining  the  calculations  involved.  One  participant  might  begin  with  the  formula  for  the 
volume  of  an  ellipsoid  and  then  assume  a  volume  for  peanuts  and  proceed.  Another  might 
begin  with  the  volume  of  an  ellipsoid  and  then  add  a  clever  correction  factor  for  the  additional 
volume  represented  by  the  pig’s  feet  and  head.  Yet  another  might  simply  use  the  volume  of  a 
sphere  but  add  an  innovative  adjustment  for  the  stochastic  nature  of  the  space  between  peanuts 
as  they  do  or  do  not  nest  well  with  each  other.  The  expert  on  mountebanks  might  not  able  to 
articulate  his  or  her  volume  calculation  well  at  all  but  might  make  two  critical  observations 
about  the  kinds  of  tricks  that  carnival  operators  are  likely  to  pull  to  make  such  estimation 
difficult — say,  inconsistent  thickness  of  the  glass  of  the  pig  or  the  use  of  peanuts  of  different 
sizes.  As  the  experts  review  the  justifications  and  calculations  made  by  the  others,  they  may 
recognize  factors  that  they  failed  to  include  in  their  own  calculations  or  come  to  understand 
that  they  have  over-  or  underestimated  some  critical  quantity.  The  revised  estimates  are  likely 


1  See,  for  example,  Carolyn  Wong,  How  Will  the  e-Explosion  Affect  How  We  Do  Research l  Phase  I:  The  E-DEL+I  Proof  of 
Concept  Exercise,  Santa  Monica,  Calif.:  RAND  Corporation,  DB-399-RC,  2003. 

2  Bernice  B.  Brown,  Delphi  Process:  A  Methodology  Used  for  the  Elicitation  of  Opinions  of  Experts,  Santa  Monica,  Calif.: 
RAND  Corporation,  P-3925,  1968,  p.  2. 
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to  be  based  on  more  complex  calculations,  be  better  calculations,  and  be  closer  to  each  other 
than  were  the  initial  individual  expert  estimates. 


The  RAND  Afghanistan  Delphi  Exercise 

The  RAND  Afghanistan  Delphi  exercise  was  an  iterative  Delphi  exercise  based  on  the  classic 
model.  It  was  completed  via  iterative  email  exchange  between  January  21  and  February  25, 
2011.  This  section  details  the  process  used. 

By  definition,  an  expert  elicitation  is  only  as  good  as  the  experts  elicited.  An  initial  list  of 
candidate  participants  was  generated  in  consultation  with  senior  RAND  managers;  by  con¬ 
sidering  authors  of  recent  books,  reports,  studies,  and  articles  on  Afghanistan;  and  in  consul¬ 
tation  with  the  editors  of  Small  Wars  Journal.  An  initial  list  of  27  candidates  emerged  from 
this  process.  Of  these  27,  15  initially  agreed  to  participate.  Of  those  15,  13  completed  the  first 
round  of  the  exercise,  with  two  declining  to  participate  when  the  details  of  the  exercise  became 
clearer  to  them.  Of  the  13  initial  participants,  11  completed  the  entire  exercise,  with  two  fur¬ 
ther  withdrawals  due  to  time  constraints  or  discomfort  with  the  electronic  discussion  format. 
Participants  included  RAND  staff  with  expertise  on  Afghanistan  or  who  had  research  experi¬ 
ence  with  or  had  deployed  to  Afghanistan,  serving  field-grade  U.S.  military  officers  with  mul¬ 
tiple  (and  recent)  deployments  to  Afghanistan,  military  veterans  who  were  current  in  COIN 
study  and  research,  university  faculty  members,  journalists,  and  experts  from  other  prominent 
think  tanks. 

A  summary  set  of  instructions  defined  each  factor  more  extensively  and  gave  general 
guidance  regarding  the  exercise.  This  guidance  included  the  following  text: 

Please  evaluate  each  factor  as  present  (1)  or  absent  (0)  in  contemporary  Afghanistan.  All 
questions  pertain  to  the  area  of  conflict  unless  otherwise  specified.  If  the  answer  is  “well, 
it  depends  on  where  specifically  you  consider — that  factor  is  present  in  part  of  the  country 
but  absent  in  another  part,”  please  make  “worst-case”  assessments.  That  is,  if  something  is 
going  well  in  RC  [Regional  Command]  South  but  poorly  in  RC  East,  make  your  scoring 
based  on  RC  East. 

A  caveat:  While  we  want  “worst-case”  assessments,  we  do  not  want  assessments  driven  by 
isolated  events.  If  a  positive  factor  is  present  over  the  vast  majority  of  the  area  of  conflict, 
score  it  as  present  even  if  there  are  one  or  two  isolated  incidents  where  the  factor  was  not 
present. 

The  iterative  Delphi  exercise  included  three  scoring  rounds  with  two  phases  in  each  round 
except  the  last,  for  a  total  of  five  phases.  In  the  first  phase  of  each  round,  participants  provided 
scores  for  each  factor,  indicating  whether  they  believed  it  was  present  (“1”)  or  absent  (“0”)  in 
Afghanistan.  In  the  second  phase  of  each  round,  participants  were  shown  their  scores  relative 
to  the  mean  scores  of  all  participants.  Regardless  of  whether  it  was  a  first  or  second  phase,  in 
all  phases  save  the  very  first  and  the  very  last,  participants  were  asked  to  justify  their  minority 
positions  (factors  on  which  they  deviated  from  the  group  mean  by  0.4  or  more)  and  contribute 
to  the  ongoing  discussion  about  the  presence  or  absence  of  the  factors. 

In  each  phase  save  the  first  (nothing  to  discuss  yet)  and  the  last  (discussion  concluded), 
participants  were  asked  to  contribute  to  the  discussion.  In  a  traditional  Delphi  exercise,  scor- 
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ers  are  asked  to  justify  all  their  ratings  or  calculations  in  the  first  round.  However,  because 
this  exercise  included  51  individual  factors  and  because  all  participants  volunteered  their  time, 
participants  were  asked  to  provide  justification  only  for  minority  positions,  lest  a  great  quan¬ 
tity  of  volunteered  time  be  consumed  generating  justifications  for  positions  about  which  the 
entire  panel  was  in  complete  agreement.  In  the  second  phase  of  each  round,  participants  whose 
score  on  a  factor  differed  from  the  group  mean  by  0.4  or  more  were  informed  that  theirs  was 
a  minority  position  and  asked  to  justify  it.  In  this  way,  the  discussion  remained  focused  on 
factors  that  were  actually  contentious,  rather  than  being  diluted  with  justifications  of  factors 
about  which  there  was  already  significant  concordance.  Scores  that  became  minority  positions 
in  subsequent  rounds  (due  to  either  changed  scores  or  movement  of  the  mean)  were  flagged  as 
newly  minority  positions,  indicating  that  a  new  justification  was  required  from  the  participant. 

After  responding  to  requests  for  justification  of  their  minority  positions,  participants  were 
asked  to  weigh  in  on  any  of  the  ongoing  discussion  of  any  of  the  factors.  Space  was  made  avail¬ 
able  for  written  rebuttals,  counterarguments,  endorsements,  and  so  on,  aimed  at  initial  minor¬ 
ity  defenses  or  at  discussion  ensuing  from  them.  No  limit  was  placed  on  the  volume  or  charac¬ 
ter  of  the  discussion,  though  participants  were  encouraged  to  be  concise.  Instructions  invited 
participants  to  refer  to  studies,  data  sets,  personal  experiences,  or  other  evidence  that  they  felt 
supported  their  positions  or  otherwise  accounted  for  their  reasoning  or  logic.  The  discussion 
was  considerable,  spanning  more  than  50  single-spaced  pages  in  aggregate. 


Raw  Delphi  Scores 

Table  A.l  presents  the  average  score  for  each  factor  across  the  11  participants  who  ultimately 
completed  the  exercise.  Because  the  scoring  was  binary  (0  or  1),  the  raw  average  can  be  accu¬ 
rately  interpreted  as  the  proportion  of  participants  who  indicated  that  a  factor  was  present  in 
their  final  scoring.  For  example,  the  first  factor  is  “COIN  force  and  government  actions  con¬ 
sistent  with  messages  (delivering  on  promises).”  The  raw  average  is  0.18,  which  indicates  that 
18  percent  of  participants  indicated  that  the  factor  was  present  (scored  it  as  “1”)  in  their  final 
scoring. 

For  reference,  Table  A.l  also  presents  the  consensus  results  (in  the  “rounded  result” 
column)  used  in  the  main  body  of  this  paper.  Recall  that  70-percent  agreement  was  the  thresh¬ 
old  specified  for  consensus.  This  means  that  raw  average  scores  of  0.7  or  higher  were  considered 
to  indicate  consensus  on  presence  and  were  rounded  to  “1”  accordingly.  Inversely,  scores  of  0.3 
or  lower  were  rounded  to  consensus  absence,  or  “0.”  Scores  between  0.3  and  0.7  were  left  as 
contentious  scores  and  rounded  to  0.5,  indicating  a  lack  of  agreement  and  neither  presence  nor 
absence. 
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Table  A.1 

Raw  Average  Scores  from  RAND  Afghanistan  Delphi  Exercise  and  Rounded  Consensus  Results 

Raw  Rounded 

Factor  Average  Result 

COIN  force  and  government  actions  consistent  with  messages  (delivering  on  promises)  0.18  0 

COIN  force  maintains  credibility  with  population  in  the  area  of  conflict  (includes  0  0 

expectation  management) 

Messages/themes  coherent  with  overall  COIN  approach  0.55  0.5 

COIN  force  avoids  creating  unattainable  expectations  0.27  0 

Themes  and  messages  coordinated  for  all  involved  government  agencies  0  0 

Earnest  IO/PSYOP/strategic  communication/messaging  effort  0.9  1 

Unity  of  effort/unity  of  command  maintained  0  0 

Flow  of  cross-border  insurgent  significantly  decreased,  remains  dramatically  reduced,  or  0  0 

largely  absent 

Important  external  support  to  insurgents  significantly  reduced  0  0 

Important  internal  support  to  insurgents  significantly  reduced  0  0 

Insurgents'  ability  to  replenish  resources  significantly  diminished  0  0 

Insurgents  unable  to  maintain  or  grow  force  size  0  0 

COIN  force  efforts  resulting  in  increased  costs  for  insurgent  processes  0.91  1 

COIN  forces  effectively  disrupt  insurgent  recruiting  0  0 

COIN  forces  effectively  disrupt  insurgent  materiel  acquisition  0  0 

COIN  forces  effectively  disrupt  insurgent  intelligence  0  0 

COIN  forces  effectively  disrupt  insurgent  financing  0  0 

Government  corruption  reduced/good  governance  increased  since  onset  of  conflict  0.09  0 

Government  leaders  selected  in  a  manner  considered  just  and  fair  by  majority  of  0.18  0 

population  in  area  of  conflict 

Majority  of  citizens  in  area  of  conflict  view  government  as  legitimate  0.27  0 

Government  provides  better  governance  than  insurgents  in  area  of  conflict  0.27  0 

COIN  force  provides  or  ensures  provision  of  basic  services  in  areas  it  controls  or  claims  to  0.27  0 

control 

Government  a  functional  democracy  0  0 

Government  a  partial  or  transitional  democracy  0.91  1 

Free  and  fair  elections  held  0.18  0 

Government  respects  human  rights  and  allows  free  press  0.45  0.5 

Intelligence  adequate  to  support  kill/capture  or  engagements  on  COIN  force's  terms  0.91  1 

Intelligence  adequate  to  allow  COIN  forces  to  disrupt  insurgent  processes  or  operations  0.91  1 

COIN  force  of  sufficient  strength  to  force  insurgents  to  fight  as  guerrillas  0.91  1 

Government/state  is  competent  0  0 
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Table  A.1 — Continued 

Raw  Rounded 

Factor  Average  Result 

COIN  force  avoids  excessive  collateral  damage,  disproportionate  use  of  force,  or  other  1  1 

illegitimate  applications  of  force 

COIN  force  seeks  to  engage  and  establish  positive  relations  with  population  in  area  of  0.91  1 

conflict 

Short-term  investments,  improvements  in  infrastructure/development,  or  property  0.91  1 

reform  in  area  of  conflict  controlled  or  claimed  by  COIN  force 

Majority  of  population  in  areas  of  conflict  supports/favors  COIN  forces  0.45  0.5 

COIN  force  establishes  and  then  expands  secure  areas  0.55  0.5 

COIN  force  has  and  uses  uncontested  air  dominance  1  1 

Perception  of  security  created  or  maintained  among  population  in  areas  COIN  force  0.18  0 

claims  to  control 

COIN  force  employs  escalating  repression  0  0 

COIN  force  employs  collective  punishment  0  0 

Primary  COIN  force  perceived  to  be  an  external  occupier  1  1 

COIN  force  or  government  actions  contribute  to  substantial  new  grievances  claimed  by  0.82  1 

the  insurgents 

Militias  work  at  cross-purposes  with  COIN  force/government  0.73  1 

COIN  force  resettles/removes  civilian  populations  for  population  control  0  0 

COIN  force  collateral  damage  perceived  by  population  in  area  of  conflict  as  worse  than  0.27  0 

insurgents' 

In  area  of  conflict,  COIN  force  perceived  as  worse  than  insurgents  0.09  0 

COIN  force  fails  to  adapt  to  changes  in  adversary  strategy,  operations,  or  tactics  0.09  0 

COIN  force  engages  in  more  coercion/intimidation  than  insurgents  0  0 

Insurgent  force  individually  superior  to  COIN  force  by  being  either  more  professional  or  0.09  0 

better  motivated 

COIN  force  or  allies  rely  on  looting  for  sustainment  0.4  0.5 

COIN  force  and  government  have  different  goals/level  of  commitment  1  1 
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